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PREFACE 


The purpose of this document is to describe the evaluation of a 
maximum likelihood classification procedure for detecting and locating 
surface water using data from the multispectral scanner (MSS) on the 
Earth Resources Technology Satellite (ERTS-1) , this activity was under- 
taken to support implanentation of Federal legislation requiring the 
inventory of inpoundments. 

This document includes background information on why the valua- 
tion was conducted; a statanent of the problem; a detailed description 
of the technical approach used; a statanent of the performanct; results; 
and reccHimendations as to the most appropriate procedure to evaluate 
next. 


This document was prepared pursuant to requiranents identified 
with the /^plications Office of the Earth Observation Division. It is 
conprised of a joint effort of personnel within the Earth Observations 
Division and personnel within the Earth Resources Department, Lockheed 
Electronics Conpany, Inc. Prime technical contribution to the effort 
described herein was made by T. C. Minter, scientific progranining 
analyst. Activities by the contractor were authorized under contract 
NAS 9-12200. 
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1 . 0 SUMMARY 


In this document, a description is presented of the results from 
a detailed evaluation of a conputer-aided procedure for processing 
EPTS-1 data to detect and locate surface water for the National Pro- 
gram for the Inspection of Dams (NPID) . The procedure w£is evaluated 
using data from a study area in the vicinity of the Lake Somerville 
area in Washington County, Texas. 

The procedure evaluated consisted of (1) selecting water training 
fields, (2) aggregating the training sanples together and clustering 
them into unimodal clusters, (3) conqjuting the mean vector and covari- 
ance matrix for each cluster, (4) classifying all of the study area 
into classes corresponding to the clusters using the rriaximum likelihood 
classifier, and (5) thresholding out the non-water pixels.^ 

Water training fields were selected from the ERTS-1 multispectral 
digital image without the aid of ground truth using a grey map of 
channel 4. This constraint and associated rationale are discussed in 
detail . 

The result of the evaluation was that the use of the procedure 
failed to provide acceptable performance results. The success criteria 
established for this study was that 90 ^ of all areas of surface water 
of 10 acres or more had to be correctly identified and located with a 
frequency of false detection of 10 % or less. The result from the pro- 
cedure evaluation was that 1001 of all areas of surface water of 10 
acres or more were correctly detected, but the frequency of false 
detection was approximately 96.81. 


pixel is the basic unit in image reconstruction from the digi- 
tal tape, using electronic display devices. It is the binary integer 
recorded on magnetic tape that represents a time san5)le of the analog 
scan line trace, the value of which is proportional to the energy 
sensed by each ERTS-1 MSS channel. 
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It was concluded from this evaluation that two principal sources 
of error existed; first, the water training field selection procedure 
potentially included pixels from the perimeter of areas of surface 
water, wet fields, and bare soil areas. The presence of the wet fields/ 
bare soil/perimeter pixels among the water training samples resulted in 
the presence of two clusters in the non-water part of feature space. 

These two clusters resulted in many false detections when wet areas 
and bare soil were classified as water. The development of a method 
for selecting thresholds wliich would eliminate most of the pixels 
assigned to these two clusters was beyond the scope of this evaluation. 
Secondly, another possible source of error was contributed because the 
ISOCLS clustering routine, as used in this evaluation, did not define 
unimodal clusters and compute meaningful statistics (mainly the covari- 
ance matrix) for each class. The classes defined by ISOCLS did not 
appear to conform to the normality assumptions and were generally multi- 
modal. No conclusion was reached as to the effect of this problem on 
classification accuracy. 

It is recommended as a follow-on to this exercise that (1) using 
representative and verified training samples for water only (i.e. water 
training samples that have been verified from photography) , a procedure 
be defined for identifying water using LARSAA (with and without the aid 
of clustering) , (2) using representative and verified water and non- 
water training samples a procedure be defined for identifying water us- 
ing LARSM (with and without the aid of clustering) , (3) using represent- 
ative and verified water and non-water training san^les, train another 
classifier (such as described in Reference 3) which is independent of 
certain of the assumptions made in Gaussian maximum likelihood classifi- 
cation (such as the normality assumption and the unimodal classes assump- 
tion) for the purpose of obtaining an independent assessment of how well 
the maximum likelihood classifier i£^ performing in meeting the assumptions 
on which the classifier is based, and (4) define a procedure for select- 
ing representative training sanples for water and non-water which does not 
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include pixels from the perimeter of areas of surface water, wet fields 
and bare soil areas in the water training sanple and does not include 
water pixels in the non-water training sample. 
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2.0 INTRODUCTION AND BACKGROUND 


In this technical report the results frc»n an evaluation of a pro- 
cedure for processing of EKTS-l data to detect and locate surface water 
in siqjport of the National Program for the Inspection of Dams (NPID) 
will be discussed. The procedure was evaluated on data from a study 
area near Lake Somerville in Washington County, Texas. In this aocu- 
ment the evaluated data processL^ig procedure will be outlined and the 
results obtained will be discussed. An analysis of the results obtained 
at each st^ of the procedure will then be given. Ccnclusions drawn 
from this analysis will be given and recomnendations as to the most 
appropriate procedure to evaluate next wil.. be given. 

For the purposes of understanding why this procedure was investi- 
gated, background related to the NPID is presented below. 

In August 1972, the President signed into law Public Law 92-367 vdiich 
authorized the Secretary of the Army to undertake a national program 
tor the inspection of dams. The need for dam safety w ught to 
national attention when water inpoundments in West Virg .x.i and South 
Dakota gave way, resulting in loss of life and propeity. 

In brief the law directs the Secretary of the Am^, acting through 
the Chief of Engineers, to carry out a national program for the inspec- 
tion of dams. The scope of water inpoundment capacity covered by the 
law is graphically described in Figure 1. To determine whether a dam 
(including the waters iitpounded by such dam) constitutes a danger to. 
human life or property, the Secretary will take into consideration the 
possibility that the dam might be endangered by overtopping, seepage, 
settlement, erosion, sediment, cracking, earth movement, earthquakes, 
failure of bulkheads, flashboard, gates on conduits, or other conditions 
which exist or which might occur in any area in the vicinity of the dam. 
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Figure 1. - Scope of water inqjoundment capacity included in Public Law 
92-367. 
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The report by the Secretary o£ the Amy to Congress is due on or 
before July 1, 1974. The report will include (1) an' inventory of all 
dams of interest located in the United States, (2) a review of each 
inspection made, the reconinendations furnished to the governor of the 
state in which such dam is located and information as to the implemen- 
tation of such recoranendations, and (3) reconinendations for a compre- 
hensive national program for the inspection, and regulaticm for safety 
purposes of dams of the nation, and the respective responsibilities 
iidiich should be assumed by the federal, state, and local ^vemments 
and by public and private interests. 

In Decanber, 1972, the Texas Water Ri^ts Coninissicn (TWRC) sub- 
mitted, throu^ the office of the Governor of Texas, a request for 
assistance by NASA/JSC/EOD in the development of a procedure or proce- 
dures for utilizing data acquired by the Earth Re ources Technology 
Satellite (EKTS-1) in detecting and locating water iipoundments. 

The EKTS-1 satellite was placed in orbit to gather data relative 
to the environment of the Earth. The EKTS-1 satellite orbits the Earth 
in a circular, sun- synchronous, near-polar orbit at an altitude of 
approximately 494 nautical miles. The satellite orbits the earth 
approximately 14 times each day and views the same scene on the earth 
approximately every 18 days. 

On board EKTS-1 is a multispectral scanner (MSS) vdiich receives 
spectral information in four channels covering the following wavelengths 


Channel 

Spectral Band 

Wavelength (micrometers) 

1 

4 

0.5 - 0.6j 

visible 

2 

5 

0.6 - 0.7' 


3 

6 

0.7 - 0.8, 

[ reflective 

4 

7 

0.8 - 1.1 

) infrared 
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The )SS data are recorded and transnitted in digital fonnat to 
the NASA data processing facility. Two image products are produced for 
each scene, in the form of photogr£q[Mc images and digital images. 

The NES products used for the procedure described in this docunent 
are: (1) 9 1/2" x 9 1/2" syston corrected i^otographic images at a 

scale of 1:1,000,000 and, (2) system corrected ccnqputer compatible 
tape (GCT) digital images. Each digital imag^ consists of four (XT's, 
each C(T covering a strip 25 by 96.3 nautical miles. 
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3.0 GENERAL DESCRIPTION OF THE PROCEDURE EVALUATED 


3.1 Statement of the Problem 

Previous attenpts to use the LARSAA naxioum likelihood classifier 
program to detect surface water using the ERTS-1 data (Reference 2) met 
with limited success. The three procedures tested cwisisted of apply- 
ing the maxiiun likelihood classifier by (a) selecting water only train- 
ing fields, clustering the training sanqples, classifying, and th^ 
thresholding, (b) selecting water training fields and a similar class 
training fields, clustering both the water and similar class training 
sanples, classifying, and then thresholding, and (c) selecting water 
only from knowr water bodies larger than 20 surface acres clustering 
the training sanples, classifying, and then thresholding- 

The results obtained from a test of procedure (a) (CLASSIFY maxi- 
naim likelihood classification using water only training fields) were 
that the correct identification of 81* of areas of surface water of 10 
acres or more, and a frequency of false detection of 82.5% were achieved. 

Procedure (b) (CLASSIFY maximum likelihood classification using 
water plus a similar class training fields) resulted in the correct 
identification of 94% of all areas of surface water of 10 acres or 
more and had a frequency of false detection of 66%. 

Procedure (c) (CLASSIFY maximum likelihood classification using 
only water bodies greater than 20 surface acres as training fields) 
correctly identified 69% of all areas of surface water of 10 acres or 
more, and had a frequency of false detection of 88%. None of these 
results were acceptable according to the established performance 
criteria. These established success criteria required a correct 
identification of 90% of all areas of surface water of 10 acres or 
more, and a frequency of false detection of 10% or less. 
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3.2 Procedure Evaluated 


As a result of the evaluation of these procedures it was concluded 
that the best approach for identifying water would be to (1) select 
water training fields, (2) Imp the training sanples together and 
cluster them into unimodal clusters, (3) conpute the mean vector and 
covariance matrix for each cluster, (4) classify all of the study area 
into thes*" clusters using the maxinum likelihood classifier and then, 
(5) threshold out the non -water pixels. 

The procedure to be evaluated is similar to one of the procedures 
tested previously, but with two inportant modifications. First, the 
clustering routine ISOCLS, was to be set up to cluster the water train- 
ing samples into a larger nianber of smaller clusters. The smaller 
clusters would help insure that the clusters were unimodal and conform 
closer to the noimality assumption. Second, a systematic approach to 
selecting class thresholds, as described in Reference 1 was to be used. 
The clustering and the threshold selection procedure will be discussed 
in greater detail later in this document. 

In testing this procedure an inportant constraii.t was imposed that 
required that all water training fields had to be identified on the 
EKTS-1 imagery without the aid of ground truth. Previous exp'-rience 
indicated that water had two important characteristics that might 
facilitate identification for training purposes. First, adjacent water 
pixels tend to have similar radiance values. Second, most water pixels 
have radiance values frcmi 0 to 6 in channel 4 and any surrounding wet 
areas generally have radiance values of 7 to 9 in channel 4. 

The water training field selection procedure then had the require- 
ment to find these hcmiogeneous areas of low radiance values of at least 
8 sanples in channel 4 on a LARSAA/PIOCN grey map and use them as 
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water training fields. To obtain a representative training sanple of 
the various areas of surface water and to avoid biasing the water class 
statistics in favor of the large water bodies (whose location are gene- 
rally known already) , the nunber of training samples taken from any one 
area of surface water was restricted to a maximum of 30 saJH)les. 

The training sanples were next aggregated together and clustered 
to obtain the water subclasses inherent in the data, by a proper set- 
ting of the clustering program's control parameters, the water training 
san 5 >les were partitioned into a large number of small clusters to insure 
the clusters were unimodal, and thus conform more closely to the estab- 
lished normality assunption which is the basis of the Gaussian maximum 
likelihood classifier. 

Next, all of the study area data was classified, using the maximum 
likelihood classifier into the subclasses defined by ISOCLS. The maxi- 
mum likelihood classification of water into the various water subclasses 
was not helpful in separating water from non-water. Thresholding was 
used to eliminate the non-water pixels from each water subclass. The 
thresholds effectively defined the discriminant boundar)' between 
water and ncn-water. The assutq)tion made in using this procedure was 
that if the water subclasses were sufficiently well separated from the 
non-water classes in spectral space, then the thresholds would be an 
effective means of defining the discriminant boundary between water and 
non-water. It should be noted, however, that this is not a maximum 
likelihood classification rule. Maximum likelihood classification 
inplies that a conditional probability density function is available 
for the non -water class and each pixel is assigned to the class (i.e. 
water or non-water) for vdiich the conditional class probability is 
greatest. More precisely, the procedure described above for identify- 
ing water is a Gaussian hypothesis testing procedure where a pixel is 
assumed to be water if its conditional probability is greater than a 
certain confidence level. However, since the CLASSIFY maximum likeli- 
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hood classifier was used to evaluate this procedure, the procedure will 
be referred to as a maximum likelihood procedure in this document. 


A systematic approach to selecting class thresholds was used, and 
is described in detail In Appendix B. This is essentially the same as 
the procedure reported in Reference 1. The procedure will be described 
briefly. 

In using the LARSAA processor routine DISPLAY, the investigator 
can specify the threshold value, T^, for each material. In the past 
this was done in two ways. 

(2) In the first proceudre the program DISPLAY vas run with 
various values of T. until the classification map (in the form of a 
line printer listing) j^eared acceptable. This procedure is very 
subjective and is difficult to repeat. 

(b) The second procedure is based on the accepted fact that if 
X (a data value) is distributed according to the multivariate normal 
distribution 



Pi(X) = 

g-1/2 Q.(X) 

(1) 


1 



idiere 

Qi(X) = 

(X - - Mj) = ZTj 

(2) 


then the quantity Q^(X) in equation (2) is a random variable having a 
chi-square distribution with N degrees of freedom (vdiere N is the 
dimension of the measurement vector X). In equation (2) is the mean 
vector, Kj, is the covariance matrix, and Tj^ is the threshold value for 
class i. To the extent that the training data is noimally distributed 
the investigator can look 15) in a cumulative chi-square table the thres- 
hold value which will reject (i.e., assign to the unclassified category) 
no more than a specified percentage of sanples. For exanple, a thres- 

3-4 



hold setting of = 3.0 vrill reject no more than 5 percent of sanples 
drawn from a two-dimensional multivariate normal distribution. 

In many cases the training data is not normally distributed and 
the distribution of must be determined en^iirically to select the 
threshold values. To enpirically form an estimate of the density 
function of Q^, the number of occurrences of each value of Q^, among 
the training sanples for class i, are counted. Then the density func- 
tions are accumulated (i.e. integrated) with respect to to form the 
distribution of for each class . Qass thresholds T^^ are then selected 
frara the en 5 >irical distribution of where T^ = l/2Qj^. 

The specific procedure evaluated will be briefly described next. 

A detailed description of this procedure is given in Appendix A. 

(1) The appropriate ERTS-1 data tape was selected and registered 
to a 1:24,000 scale map of the test area using the program REGSTR. 

(2) The PICMON program was used to obtain a grey map of channel 4. 
On the grey map counts 0 to 6 were represented by the symbol M, counts 

7 to 9 an asterisk (*), and counts 10 to 63 by a blank (no symbol). 

(3) From the PICJO^ map, all areas were located that contained 
eight or more contiguously printed M or * symbols , and where at least 
25% of the symbols were M symbols. Within these areas, the largest 
possible rectangle containing at least 50% M syniiols was inscribed. If 
the resulting rectangle contained less than eight symbols (including M 
and *), the area was deleted from further consideration. If the rectangle 
consisted of more than 30 symbols (M's and **s), the size of the rectangle 
was reduced in size, to contain no more than 30 symbols. 
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(4) All of the training sanples obtained in the step above were 
aggregated together and clustered with ISOCLS, using channels 1 and 4 
withMAXCLS= 20, N4IN = 16, STDMAX = 1.5, DLMIN = 3.2, and ISTOP = 20. 
Statistics were then obtained for each cluster. 

(5) The study area near Lake Somenrille was classified using 
channels 1 and 4 and using the cluster statistics obtained ir^ step 4. 

(6) The density and cumulative distribution of the quadratic form 
were estimated for each subclass. Threshold values were obtained from 
these cumulative distributions of the quadratic form (see i^pendix B) . 

(7) DISPLAY was ixin with the threshold values obtained above in 

step 6. 

(8) On the resultant display map, all areas representing class 0 
(non-water areas of 10 surface acres or more improperly classified as 
water). Class III areas (10 or more surface acres), were located and 
evaluated against existing ground truth data. The established perfor- 
mance criteria A>r this evaluation is discussed in detail in i^pendix 

C. 


In the following sections, the results of the evaluation of the 
procedure just discussed will be documented. The analytical approach 
used to obtain these results will then be discussed. Conclusions arriv- 
ed at will be presented, and recommendations for further actions will be 
presented. 
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4.0 RESULTS, CONCLUSIONS, AND RECOMMENDATIONS 


4.1 Performance Test Results 

The procedure described in section 3.2 was used to classify EKTS-l 
data (ERTS-1 Scene: E-1092-16305) acquired on October 23, 1973. Appro- 

ximately 500 areas were identified as Class III areas (refer to Appendix 
C) . Of the 16 Class III areas determined from photographic ground truth 
(Mission 220, flown 8 November 1972), all such areas corresponded to 
identified Clciss III areas on the data display map. However, approxi- 
mately 484 of the identifications of Class III areas corresponded to 
areas where no surface water was found to exist from photointerpretative 
"ground truth". Thus by the definitions of the performance criteria in 
^pendix C, the percentage of Class III areas correctly located by this 
procedure was 

The frequency of false detection ^03 was, however, 

P _ 484 05 , 

% 16+484 

> 

Therefore, the chosen procedure failed to meet the criteria of a freq- 
uency of false detection of 10 % or less, though it exceeded criteria of 
the correct identification of Class III areas of 90% or greater. 

4.2 Analysis of the Results 

Seven procedural steps were taken to develop the analytical results 
The results of each step of the procedure described in the previous sec- 
tion are presented here along with an analysis of each step. 


4-1 



(A) Step 1 - An ERTS-1 data tape (ERTS-1 Scene: E-1092-16305) of 

the Lake Somerville Study Area was registered to a scale of 1:24,000 
using REGSTR. A pixel dropout rate of approximately 25% was experienced 
in registering these data. This dropout rate may result in Class III 
areas being classified as Class I or II areas. 

(B) Step 2 § 3 - These two steps involved the random selection of 
20 water training fields. Only 20 fields were selected because this 
number is the niaxinum number of distinct rectangular areas that ISOCLS 
can process at one time. 

(C) Step 4 - ISOCLS was used to cluster the 248 pixels from the 
20 water training fields into 9 clusters, and statistics were defined 
using ISOCLS. The scatter plot in Figure 2 weis conpited for channels 
1 and 4. The mean for each cluster is shown on the scatter plot in 
Figure 2. Table 1 lists the mecins vector and covariance matrix com- 
puted for each cluster. 

On the basis of the scatter plots, no conclusions could be made 
as to the adequacy of I^X!LS for defining clusters and cluster statis- 
tics that could be used in a Gaussian maxinum likelihood classification. 
In Figure 2, it can be noted that the cluster means generally fall near 
the modes of the data. A possible problem is noted with the covariances 
computed for some of the clusters: Clusters 1, 3, 6, 7, 8, and 9 had 

negative covariances. Negative covariances are normally not present 
in distributions encountered in remotely sensed data. The presence of 
negative covariances accentuated the problem of selecting thresholds. 

The negative covariances effectively moved the wet areas and vegetation 
closer to the neans of the water clusters and made thresholding more 
difficult. 
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Figure 2. - Scatter plot of water training smples. 
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TABLE 1 - CLUSTER STATISTICS 


MEANS 


CLUSTER 

CH{1) 

CH(4) 

1 

43.20 

5.05 

2 

22.05 

5.98 

3 

21.63 

3.19 

4 

24.32 

1.96 

5 

18.64 

.84 

6 

36.50 

5.17 

7 

28.07 

7.64 

8 

32.96 

3.19 

9 

27.24 

3.48 


COVARIANCE MATRIX FOR CLUSTER 1 
10.96 

-3.21 3.55 


COVARIANCE MATRIX FOR CLUSTER 4 
.93 

.15 .39 


COVARIANCE MATRIX FOR CLUSTER 


COVARIANCE MATRIX FOR CLUSTER 5 


.63 

.10 .47 


30.39 

1.26 .45 


COVARIANCE MATRIX FOR CLUSTER 


COVARIANCE MATRIX FOR CLUSTER 6 


1.12 

-.01 .37 


4.25 

-2.08 2.64 


COVARIANCE MATRIX FOR CLUSTER 7 


2.64 

-.40 1.23 


COVARIANCE MATRIX FOR CLUSTER 8 


.85 

-.47 .74 

COVARIANCE MATRIX FOR CLUSTER 9 


1.03 

-.03 .55 

TOTAL NUMBER OF CLUSTERS • 9 
TOTAL NUMBER OF POINTS « 248 


CLUSTER SYMBOL POINTS IN CLUSTER 


1 

2 

3 

4 

5 

6 

7 

8 
9 


1 

2 

3 

4 

5 

6 

7 

8 
9 


20 

62 

27 

28 
25 
12 
14 
27 
33 
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The most critical problem depicted by Figure 2 is the presence o£ 
a large nunber of pixels from the perimeter of areas of surface water, 
wet areas, and bare soil vdiich are incorrectly included in water train- 
ing fields by losing the water training samples selection procedure de- 
scribed in Step 3 of the procedure. 

On reviewing the water training fields within the Lake Somerville 
study area, it was found that approximately 40® of the water training 
fields selected corresponded to wet fields oj bare soil areas as deter- 
mined from photointerpretation "ground tr The criteria used in 

Steps 2 and 3 also include what is belie' he some pijv,.is related 

to the perimeter of an area of surface wat ..j.. These pixels are parti- 
tioned into clusters 2 axud 7. This was also evident from the scatter 
plot in Figure 2. Many of these wet area/bare soil/perimeter pixels 
fall on the non-water side of the Two Channel Linear Discriminant 
Boundary shown in Figure 2. This is an empirically derived discriminant 
boundary (see Reference 2) which has been shown to provide acceptable 
classification' results (as described in Section 3.0, Appendix C), using 
ERTS-1 data for East Texas. Additionally, in subsequent classification 
results (see Step 5), these perimeter pixels, wet areas, and bare soil 
areas were identitied as belonging to clusters 2 and 7. 

(D) Step 5 - The Lake Somerville study area was classified using 
channels 1 and 4 and the cluster statistic obtained in St'-.p b. \ LARSAA 
map tape (N^VPTAP) was generated. 

(E) Step 6 - The threshold values for each of the 9 subclasses 
were selected. An empirical threshold selection procedure (described 
in Appendix B1 , was used to select threshold values . This procedure 
provides a method for obtaining thresholds even when the classification 
data does not confoim to the normality assurn)tion. 


4-5 



The procedure described in Appendix B was used to form an empirical 
estimate of the density function of (see equation (2) page 2-4 by 
counting the number of occurrences of each value of Q. among the train- 
ing sait5)les for class i. The density function of for each of the 
nine water si'^classes are shown in Figure 3. 

The density functions of were then summed with respect to 
to form the distribution of for each of the nine water subclasses 
as shown in Figure 4. 

In addition, an enpirical estimate of the density function of 
was formed by counting the number of occurrences of each value of 
among all of the pixels eissigned to each of the nine water subclasses 
in the Lake Somerville study area. These density function estimates 
for the nine water subclasses are shown in Figure 5. 

Two conclusions can be drawn fran data presented in Figures 3 and 
5; First, in Figure 3, tne density and distribution function estimates 
of Q^(X) of the training samples for each of tne nine water classes 
differed significantly from the theoretical chi-square density function 
and the theoretical cumulative chi-square distribution function shown 
in Figure 6. It can be concluded from the density function estimate 
of (Figure 3) for the training sanples that in general the water 
class training san5)les are multimodal and do not conform tc the normality 
assunption. The multiraodality of the water training sanp' rs could possi- 
bly be attributed to the small number of training samples used to foim 
the estimate of Q^, but in Figure 5 the density functions of of all 
of the pixels assigned to each class in the Lake Sonerville Study Area 
also differ from the tlieoretical chi-square density for low values of 
(i.e. for values of less than approximately 3 to 5) . Since the 
non-water classes (i.e. vegetation, wet area, etc.) appear out in the 
tails of the density function of (i.e. for values greater than 
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Figure 3. - Continued. 
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Figure 4. - Continued. 
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Figure 5. - En^irical estimate o£ the density functions of the quadratic 
form for the nine water classes - estimated using all of the 
pixels^assigned to each class in the Lake Somerville study area. 
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THEORETICAL CHI-SQUARE DENSITY FUNCTION 
FOR TWO CHANNELS 



Q — 

THEORETICAL CUMULATIVE CHI-SQUARE DISTRIBUTION 
FUNCTION FOR TWO CHANNELS 


Figure 6. - The theoretical chi-square density and cumulative distri- 
bution for two degrees of freedom. 
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approximately 3 to 5), it would not be unexpected if the tails of the 
density of differed significantly from the theoretical chi-square 
density. From the data shown in Figures 3 and 5, it can be concluded 
that the clusters defined by IS(X1S are generally multimodal and do not 
conform to the normality assumption. In addition, a comparison of the 
forms of the density functions of for low values of 0^^ (i.e. for 
values less than approximately 3 to 5 in Figures 3 and 5) on a class - 
by-class basis indicated that the training sanples for each class are 
not generally representative of the classes of water found in the Lake 
Somerville Study Area (class S is an exan^jle of a class for which the 
training sanples ^ appear to be representative of that class over all 
the study area even though it is multimodal). Under these conditions, 
the procedure adopted for selecting thresholds was as follows: 

(a) Using the estimate of the cumulative distribution function of 

Qj^ obtained from the training samples and shown in Figure 4, determine 
what threshold value (i.e. = 1/2 Q^) is required to retain some 

initial percentage of the training samples between 9C% and 100%. These 
bounds are inposed by the performance criteria discussed in Appendix C. 

An initial value of 95% was selected as the retention rate for testing 
this procedure. 

(b) Run DISPLAY with the threshold value selected above. 

(c) Evaluate the performance of the classifier using the perfor- 
mance evaluation criteria described in Appendix C. 

(d) If the performance of the classifier is acceptable, the thres- 
hold selection procedure is conpleted. Otherwise go back to (a) above 
and pick new thresholds based on the result of the performance evalua- 
tion (i.e. if more than 90% of all areas of surface water of 10 acres 

or greater in size were correctly detected, with a frequency of false 
detection greater than 10%, then lower the training sanple retention 
rate to a value nearer 90% or vice versa). 
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(F) Step 7 - DISPLAY was run with the threshold values obtained in 
step 6 to display the results contained on the MAPTAP obtained in step 5. 

(G) Step 8 - Using the display map it was estimated that there 
were approximately 500 class 0 areas (i.e. non-water areas misclassified 
as Class III areas). The frequency of false detection was approximately 
96.8%. Areas of surface water of 10 acres or more (Class III areas) 
were detected and located with an accuracy of 100% . Areas of surface 
water from 7 to 9.9 acres (Class II areas) were detected and located 
with an accuracy of 20% . All Class II areas were detected but many 
were represented by more or less than two line-printer symbols. An 
Einalysis of the display map indicated the major source of error was 

the misclassification of wet areas, bare soil, and perimeter pixels 
as water. These pixels were generally associated with classes 2 or 7. 
Figure 7 shows the outline of a 28.9 acre area of surface water on the 
display map classification results. Many of the pixels around the 
perimeter of the lake were identified as belonging to class 7. An 
analysis of five of the Class III areas indicated that when consider- 
ing all of the perimeter pixels (both thresholded and unthresholded) , 

65% of the perimeter pixels were thiesholded, 33% belonged to class 7, 
and 2% to class 2. Using the overlays for the perimeter of areas of 
surface water vdiich were generated from photography (Mission 220) , it 
was estimated that within the boundaries of the Class III areas (10 
surface acres or more), 90% of the pixels belonged to classes 3, 4, 5, 

6, and 9. Nine percent of these pixels belonged to class 7, and 1% to 
class 2. Of the pixels that lie on the perimeter of the areas of sur- 
face water and were not thjresholded, 70% belonged to class 7, 24% to 
class 2, and 6% to class 1. 

The mislabeling of a perimeter pixel as water does not effect the 
accuracy with which Class III areas are detected. However, it does 
increase the possibility that a Class II area will be incorrectly 
identified as a Class III area. Instead, the major source of error is 
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Figure 7. - Classification results for a 28.9 acre area of surface water. 
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the false detections which occur when wet areas and bare soil are mis- 
labeled as water. Among the wet areas and bate soil areas identified 
from photography, it was found that in ehe wet areas 531 , of the false 
detections belonged to class 2 , and M% to class 7 , and in the bare 
soi] areas, 63% of the false detections beloiiged to class 7, and 37% 
to class 2. None of the other classes were present in these areas. 

One ccxnmercial area, which was falsely identified as water, had 2 oixels 
from class 6, and 3 pixels from class 7. 

As a result of this analysis, it was concluded that classes 2 and 
7 were the major sources of false detections. To eliminate the false 
detections introduced by these two classes, nearly all of the pixels 
assigned to these two classes have to be thresholded out. If all of 
the pixels assigned to these tvro classes had been thresholded out 
however, one of the Class III areas would not hrve been correctly 
identified, two Class III area, would have been r.isclassified as Class 
II are:is, and the remaining 13 Class III areas in the Somerville sti.dy 
area would have been correctly identified. 

Since the preceding discussion has shovi that 'lusters 2 and 7 
were the results of incorrectly identified training fields, steps 2 
and 3 of the procedure n' ;d to be changed, not the thresholds . Thres- 
holding all of the pixels assigned to class 2 and 7 implies some a 
priori information about these two classes. This does not appear to 
be a reasonable approach to eliminating false detections introduced 
by these two classes, so no other threshold values were tried for the 
purpose of this study. 
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4.3 Conclusions 


Several problem areas were noted as a result of the analysis 
performed during this evaluation, and warrant coninent: 

(1) The water training field selection procedure incorrectly 
included pixels from the perimeter of areas of sijtrface water, wet fields, 
and bare soil areas. The presence of the wet fields/bar'> soil/perimeter 
pixels among the water training samples resulted in tr presence of two 
clusters (clusters 2 and 7 - see Figure 2) in the non-water part of 
feature space. These two clusters resulted in many false detections 
whon wet areas and bare soil were classified as water. Within the 
scope of this effort no practical method was developed for selecting 
thresholds ^ich could be used to eliminate most of the pixels assigned 
to these two clusters. 

(2) Another source of error can be attributed to the fact that 
ISOCLS, as used in this evaluation, did not define unl’.odal clusters 
and compute meaningful statistics (mainly the covariance matrix) for 
each class. The classes defined by ISOCLS did not appear to cmform 
to the normality assvinption and were generally noiltimodal. No conclu- 
sion could be reached as to the effect of this statistical problan on 
the performance results. 


4 . 4 Recommendations 

It is reccmmended that the following approach be taken in address- 
ing the detection of surface water. 
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(1) Using representative water training data as derived from 
photogrs^hic "ground truth", define new procedures for identifyiiig 
water using LARSAA: One procedure using clustering, and one procedure 
without. Select thresholds using the procedure described in Reference 
1 . 


(2) Using representative water training samples and rq[>resentative 
non-water training samples (i.e. training samples from wet areas, bare 
soil, vegetation, perimeter of areas of surface water, cloud and terrain 
shadow, etc.) that have been verified from photography, define a proce- 
dure for identifying water using LARSAA: One procedure using cluster- 
ing and one procedure without. Select threslwlds using the procedure 
described in Reference 1. 

(3) Using the water and non-water training samples used to test 
LARSAA in itemi (2) above, train another available classifier (Reference 
3), that is independent of some of the assumpticms made in Gaussian 
maxiimin likelihood classification (such as the normality assimption 
and the unimodal class assumption) for ^he purpose of obtaining an 
independent assessment of how well the maximiim likelihood classifier 

is performing in meeting its assuiptions. 

(4) Define a procedure for selecting representative training 
samples for water and non-water which do not contain mislabeled train- 
ing samples. 
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A DETAILED DESCRIPTION OF THE EVALUATED COMPUTER-AIDED DATA PROCESSING 

PROCEDURE 

This procedure involved the following functional steps: 

(1) Select the appropriate ERTS-1 System Corrected Cc*H)uter 
Conpatible tape*. 

(2) Using program REFOI^, convert the tape to a fonnat ccHH)atible 
with the classifier input (in this case LARSYS II). 

(3) Using program PlOtW, obtain a grey map of channel 4 (Caution: 
use the default mode for symbol selection.) 

(4) Select the appr iate base maps and select ground control 
points. Relate these groi J control points to line and column locations 
on the channel 4 grey map. (Reference 4) 

(5) Using program REGSTR, obtain a geometrically corrected 
LARSYS II tape. 

(6) Using program PICNDN, obtain a geranetrically corrected grey 
map of channel 4. Use the following symbols for the aj^ropriate count 
range. 


Symbol Counts 

m 

* 


blank 


A-1 


0 thru 6 
7 thru 9 
10 thru 63 



(7) From the PIODN map obtained in step 6, locate all areas 
containing eight or more contiguously printed M or * syitbols. If an 
area consists of at least 25% M symbols, outline this area and retain 
for st^ 8. 

(8) Inscribe the largest possible rectangle, vdiidi contains at 
least 50% M syrabols, within each of the areas located in step 7. If 
such a rectangle contains less than eig^t symbols (including M and *) , 
delete this area frcHn any further consideration. If the rectangle 
consists of more than 30 symbols, reduce the size of the rectangle so 
it contains no more than 30 symbols. 

The procedure for selecting water training fields as outlined in 
steps 7 and 8 above was arbitrary. 

(9) Punch LARS - 12 cards : ’r each of the fields selected in 
step 8. 

(10) Run ISOCLS using channels 1 and 4, with MAXCLAS ■ 20, 

NMIN = 16, STDMAX = 1.5, DLMIN = 3.2, and ISTOP = 20. Obtain a 
statistics deck. 

(11) Run CLASSIFY, using channels 1 and 4 and the statistics for 
the clusters frcrni step 10 and obtain a I^PTAP (map tape) . 

(12) Obtain the density and cumulative distribution of the quad- 
ratic foim for each subclass of water, and select threshold values. 

(13) Run DISPLY with the MAPTAP obtained in step 11 and the thres- 
hold values obtained in step 12 and obtain a line-printer output. 

(14) On the display map, the water classes are displayed as inte- 
gers and non-water classes are displayed as blanks. 



(15) View the display map to determine the areas on the map irdiich 
correspond to the following definitions: 

Class I areas - A classification map eirea containing one ADP sym- 
bol will be defined as the ADP identification of a lake of 2 to 6.9 
surface acres. 

Class II areas - A classification map area containing two contigu- 
ous ADP symbols will be defined as the AIF identification of a lake of 
7.0 to 9.9 surface acres. 

Class III areas - A classification map area containing three or 
more contiguous ADP symbols will be defined as the AK* identification 
of a lake of 10 or more surface acres. 



APPENDIX B 


DETERMINING THE EMPIRICAL DISTRIBUTION 
OF THE QUADRATIC FORM FOR USE IN 
THRESHOLDING 



APPENDIX B 


IffiTERMINING 'fflE EMPIRICAL DISTRIBUTION OF IHE (QUADRATIC FOFM FOR USE IN 

THRESHOLDING 

An aipirical threshold selection procedure described in Reference 
1 was used to select threshold values. This procedure provides a method 
for obtaining thresholds even \dien the class data does not conform to 
the normality assumption. This procedure will be briefly described in 
the following paragraphs and is essentially the same procedure as des- 
cribed in Reference 1. 


The LARSAA classification processor, CLASSIFY, classifies measure- 
ment vectors using a maximun likelihood scheme based on an assumed 
multivariate normal probability density function. In the case of LARSAA, 
the classification and the confidence level for each pixel are stored on 
the output map tape by the program. 


Equation 1 gives the assumed condj.tional probability density 

T 

function for the multispectral brightness vector X = (Xj^ > ^2 ’ * * * ’ 
x^, ..., x^) measured when material of class i fills the scanner field 
of view. 


Pi(x) 


(2^) 



-0.5(X - MpV'^CX - Mp 


( 1 ) 


Here M- and K. are the previously ccanputed mean vector and covariance 
matrix for the i^ class. Because of the exponential foim of equation 
1 and because ln(P.) is a monotonically increasing function of P. it 
is convenient to define a new variable according to equation 2. 

V. = ln(P.) (2) 
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Equations 3 and 4 result frcm ccxnbining equations 1 and 2 

Vi = Ci - ^ (X - - Mp (3) 

vdiere 

^i ~ - ^In^jx^jj (4) 

To select the material with the maxiirt m likelihood , the following 
decision rule is used: 

T 

A data vector X = (Xj^, X 2 » x^, 3^) is classified as belong- 

ing to class i, if, 

> Vj for all i j (5) 

Equations 3, 4, and 5 are the means by vdiich the N measurements in X 
are classified as representing a particular material -type on the basis 
of the input mean vectors and the input covariance matrices K^. 

After classification, the display processor, DISPLAY, reads and 
displays the results from the map tape (M\PTAP) and sunmarizes the 
results. This program has a provision for assigj'ing samples to the 
unclassified category if the confidence level does not exceed a value 
depaident on the specified threshold T^^; this condition is given by 
equation 6. If 

P^(X)< |max(Ppj e‘^i (6) 

then X is not assigned to a category, where max(P^) , given by equation 
7, is the maximum value of P^. 

max(PJ = max P. (M.) = max exp(V.) (7) 

1 ill . 1 
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By combining equations 1, 6, and 7 the condition for assigning a 
saitple to tne unclassified category can be expressed by equation 8. 

Q.(x) = (X - > 2T. (8) 

The value of the quadratic form, Q. , is often called Mahalanobis' 
Distance; it is the weighted (by ) squared distance in N- space 
between a given measuranent vector and the mean vector for class i. 
According to equation 8, a sample is left unclassified if its Mahalan- 
obis' Distance exceeds the value 2T^. 

In using the LARSAA program, DISPLAY, the investigator can specify 
the threshold value, T^, for all materials. In the past this was done 
in two ways. 

(1) The DISPLAY program was used with various values of T^ to 
obtain a classification map (in the form of a line-printer listing) 
which was subjectively determined to be adequate. This meant that most 
of the training and test fields were correctly classified and yet almost 
no classifications were made vdiere the mateiial type was known to differ 
from those materials defined by input mean vectors and covariance 
matrixes . 

(2) It is well known that in theory if P(x) is normal then the 
quantity in equation 8 is a random variable having a chi-square 
distribution with N degrees of freedom (wliere N is the dimension of 
the measurement vector X) . To the extent that the training data is 
normally distributed, the investigator can look up in a cumulative chi- 
square table the threshold value which will reject (i.e., assign to 
the unclassified category) no more vhan a specified percentage of sam- 
ples. For exanple, a threshold setting of Tj^ = 3.0 will reject no more 
than 5 percent of san^iles drawn from a two-dimensional multivariate 
normal distribution. 
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Since the training data is not normally distributed in many cases, 
the distribution of must be determined enpirically to select the 
threshold value. To empirically determine the thresholds for the nine 
water classes, the LARSAA map tape (MAPTAP) generated in classification 
together with a card deck giving ground truth information for the 
retangular water training fields were input to the Density and Distri- 
bution Function Program to cai5)ute the actual distribution of Q^. 
Appendix D contains a listing of the Density and Distribution Program. 
The LARSAA (MAPTAP) contained the class constants C. for each class, 
and for each pixel: 

(1) The integer value for i, the most likely material. 

(2) The floating-point variable define by equations 2 and 1 
for each of the 439 elements in each of the 1100 lines on the map tape. 

Figure B1 shows the flow of information involved in con 5 >uting the 
density and distribution functions. For each scan line, 439 values of 
i and V. are read into core. Each element is considered to determine 
whether or not ground truth is available (i.e., if it lies within one 
of the rectangular fields defined by the Ground Truth deck) . If a 
pixel lay within one of the rectangular training fields defined by the 
Ground Truth deck, the value is used with the class constant (C^ for 
that particular material) to determine the Mahalanobis' Distance. 

The nunber of occurrences for each value of for each material are 
accumulated to form an estimate of the density function. After all 
sanples on the input map tape have been tested, the density functions 
are accumulated (i.e. , sunned) with respect to to form the distribut- 
ion function for each material. The onpirical density and distribution 
functions are both printed out as functions of for each of the nine 
materials . 
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Figure B2 shows the flow of infoimation involved in ccMnputing the 
density and distribution functions vdien the map tape contains only the 
classified water training fields. This modificatiw to the Density and 
Distribution Functions Program allowed water training fields vhich were 
outside the Lake Sonerville Study Area to be included in the coii^)utation 
of the density and distribution for the quadratic form Q^(X) for each 
water class. 

In addition, the Figure B2 flow was used to obtain the density 
function and distribution function for the quadratic form for each 
class using all of the pixels in the Sanerville study area vhich were 
assigned to each class. To accomplish this, the vhole Lake Somerville 
Study Area was treateu as a single water training field in the Figure 
B2 flow. 
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LARS PROGRAM CLASSIFY 


CARDS WITH GROUND TRUTH 
INFORMATION (TRAINING 
FIELD DEFINITIONS): 

LB BEGINNING SCAN LINE 
LE - ENDING SCAN LINE 
£B - BEGINNING ELEMENT 
EE • ENDING ELEMENT 


I 


OUTPUT MAP TAPE 
CONTAINING: CLASo 
CONSTANTS. C(I>, 

1 < I < 9 SEE.EQ. 4. 

I - NUMBER OF MATERIAL 
HAVING MAXIMUM 
LIKELIHOOD 
V ■ NATURAL LOG OF 

PROBABILITY DENSIT) 
FUNCTION; SEE EQS. 

1 < 2 

FOR 1 < E<439 ELEMENTS 
IN EACH OF 1 < L< 
1100 SCAN LINES 


I 


DENSITY AND DISTRIBUTION FUNCTIONS PROGRAM: 


CONSIDER SAMPLE (L.E) M 


I 


IS LB < L 1 LE .\ND 
EB < E < EE 

FOR A PARTICUUR TRAIN- 
ING FIELD 


SET M 


I 


YES 


MATERIAL TYPE I 
OF SAMPLE (L.E) 


I 


YES 


DETERMINE QUADRATIC FORM 
ACCORDING TO EQS. 8 B 3 
Q(I) - 21C(M^)'V(L.E)1 
IQ ■ INTEGER Q(I) 


FORM PRCBABILITV DENSITY 
FUNCTION FOR: 

MATERIAL - 

MAHALANOBIS* DISTANCE • IQ 
DFdQ.Mp) - DF(IQ.Mp) ♦ 1 


IS (L,:) THE LAST SAMPLE 


NO 

UPDATE L AND/OR E 


FORM DISTRIBUTION 
FUNCTION FOk: 

MATERIAL - M 

MAHALANOBIS’ DISTANCE • IQ! 
ADF(IQ*l,Mp) • ADF(IQ.Mp) 

♦ DF(IQ.Mp) 


LINb-PP.lNTER LISTING 
GIVING 

(1) DENSITY FUNCTION FOR 
NINE MATERIALS 

(2) DISTRIBUTION FUNCTION 
FOR NINE MATERIALS 


Figure Bl. - Flow diagram for con^juting density and distribution func- 
tions for the general case idien training fields lie within the bounds 
of the area classified on the MAPTAP. 
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Figure B2. - Flow diagram for coin^'uting density and distribution func- 
tions when MAPTAP contains only the classified watc;r training fjclds. 
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APPENDIX C 


A DETAILED DESCRIPTION OF THE PERFORMANCE EVALUATION PROCEDURE 

The performance of the procedure described in Section 4 . 1 vrais 
evaluated using photointerpretation data along with the following 
evaluation rationale. The procedure is acceptable if it meets or 
exceeds the following criteria. 

(a) Detect and locate all Class III areas with an accuracy of 90% 
or greater. 

(b) Frequency of false detections of 10% or less on Class III 
areas. 

Items (a) aiiJ (b) correspond to F-j and Fqj of the matrL below. 
The remaining monbers of this matrix were not evaluated for this proce- 
dure.* 


"*inis nas been done for other procedures tested. (See Reference 2.) 



PERFORM/\NCE M/O'RIX 


Assigned Class 


True 


Class 

I 


III 

I 

*^11 

^12 

Fi3 

11 

^21 

*^22 

*^23 

III 

^31 

^32 

*^33 

0 

^01 

^02 

PQ3 


= frequency with which AW> identification of class j areas were 
actually class i areas (i.e. class j areas mis -identified as 
class i areas). 



= frequency with vdiich ADP identifications of class j areas were 
actually not an area of any class. (Frequency of False Detection) 



Lj = the total number of correct ADP identifications of class i areas. 

= the total number of class i areas in study area. 

Nj = the total number class j areas. 

= nunfcer of ADP identifications of class j areas which were actua- 
lly class i areas. 

K • = mii4>er of ADP identifications of class i areas vdiich were actua- 
oj 

lly not an area of any class. 
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APPENDIX D 


THE DENSITY AND DISTRIBUTION PROGRAM 
COMPUTER LISTING 

The following program was used to canipite the density and distri- 
bution of the quadratic form Q(x) . It is written in FORTRAN IV and uses 
approximately 500 words of core storage for code and approximately 6,000 
words for data. To process 20 training fields of water it takes approxi- 
mately 30 seconds of CPU time on the Univac 1108 EXEC 2. 
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lEPRODUcroiLiry op th^ 

ORIGINAL PAGE IS POOR 


loa 

101 

102 
lOJ 
io<« 
los 

106 

102 

iO« 


iS/aiJou?'' 

SL“f~ 

its. SA.JlS) 
f^OKMAniHl, 10113, 
fO♦<MAT^8^ 10.2, 
rUwMAT , 1H0.8F10.2, 

FOKMa 1 I 1 mO » 1 0 1 1 3 , 

FU.<«AT(,rio.lOC13.*.l 

f OKflAT , 1 M0*«, 1 J j, 

FaWMATllHlOllo.iriQ.,, 

K£ AD ( 1 , Ddm 

HtADd, (»COVfiTX(i I. I , 

sr 

6«<1 T£« 6, liiH, (cO,4S( 1 , tsi ^r.l, 

iru»?.NL.o, „o 10 si' 


1*0 


22 


101) ( 
I 1U2M 


30 


#*' 1 I fc. ( 6 , 

CONTINUE 
00 22 !■ 1 tNCLS 
NTKYl 1 ,«0 
CONTINUE 
M»0 
NT»0 

00 1 UiiUCLS 

OO 2 Uej ^ J0Q 

N0( I , jJsO 
CONTI NUE 
CONTI NUE 
CONT I NUE 

RC AD ( 1 , 1 1' r$ . L I NES 

IF • IHTS.Eg.o , 00 TO 

«£Ao«J) LdlAUJ.K- 
IFlINO.NE.O, GO TO 
ifil.lt.ned, go To 

CONTI NUE 
00 30 K«i , IPTS 
lO ( A , sQ 

CONTINUE 
DO 31 N«1 ,NTST 

12 a.LT,La(N,,OK.L.GT.L£(N) , 


! 1 1 'ti 1 1 J ; N^ ( i J ,*N H ; i ; • i * « * n tst i 

S«»» » ) .NjEl 1 , ,l»l,N^j^, 


l»lPTS),iV(K,,K«,^j PT^ I 
S 1 
3 


GO TO 31 
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Heproducibility of the 

ORIGINAL PAGE IS POr)B 


C* MATPaMATCO 
I SosNbb < N ) 

I jL = f<St I li ) 

DO 5 K»Ibt)ilst 

! U ( ro s I A ( N ) 

b continue 

31 continue 

bl CONTINUE 

00 32 K»Ii1PTS 
!l= I A ( K| 

If C IND«NE«u» oO to 52 

N® 1 b ( A » 

irlrt.NE»N» GO To 33 
bZ CONTINUE 

(^s2»0*(C0ri*SlM}«V(p. } }♦! tS 

1 0 » I N T ( 0 ) 

If ( IU.GT»0I go to II 
1W::99 

60 TO 12 

11 COi^lNUE 
lf(lO»tTi*/9) GO To 12 
IU®lUO 

12 c^ONTINUe 

NO IK, lWl=N0(i1,10)^i 
33 CONTINUE 

32 CONTINUE 

If I IND.NL»01 GO TO S3 
lf<w»£W»NLL> GO TO 6 
S3 CON.'wjE 

KEaO(i« L*nA(Kl,K=l,|PTS),(v(A»,K«l,|PTSI 
If ( lNO»NE.,o,AND.t,Ej,OJ GO TO 3 
IF(L«EQ*ni 60 TO 6 

If ( 1 n.o.ne.01 go to si 

GO TO H 
6 CONTINUE 

00 7 H«l,r,CLS 
N r®o 

00 8 ja I , 100 

NaNOl H, J» 

NP( J)*N 
N T a N T ♦ N 
0 CONTINUE 
BafLOATINT) 

OA»o » Q 

DO 9 ja 1 , 1 00 
N»NP ( Jl 
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REPRODUCroiLITy OP THE 
ORIGINAL PAGE IS POOR 


T«T tOAT ( N) 


0»T /6 


DP ( Jl *0 


UAaUA-*^0 


OfA ( J) aOA 


V CON 1 i NUC 


»;Ki TE ( 6 • lot) 1 

M * N r 

•') ft 1 T L ( 6 • 1 0 a ) 

np 

•V ^ t T {I ( 6 • t 0 tt I 

M# NT 

NK i It ( 6 , 1 .09 ) 

DF 

‘■•M Tt t 6 , »0(j j 

n , N T 

*1 •' 1 T E ( 6 , ! 0 6 ) 

0 ^ A 


7 continue 

srop 

LNU 
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